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Abstract 


Artificial Intelligence (Al) has been widely-heralded as a revolutionary technology across a 
range of domains, including defence. In this sphere, its potential for automating low-cost and 
low-risk forms of warfighting known as “remote warfare” has given rise to fears of Lethal 
Autonomous Weapons Systems (LAWS). While the use of LAWS has been subject to 
intense speculation, far less attention has been paid to their development. Notably, 
cutting-edge Al innovation is not to be found in government-funded research labs or 
traditional defence contractors, but consumer technology companies. Because of this, 
unique controversies have arisen over its development. In the case of Project Maven, a US 
Department of Defense initiative seeking to leverage Al for automating drone footage 
analysis, employee protests eventually forced one of its contractors, Google, to end their 
involvement. Using Project Maven as a case study, this research therefore seeks to 
understand the development of an emerging Military-Technological Complex developing 
these technologies and an oppositional Civil Society Coalition seeking to regulate them. 
Based on documentary analysis of key texts and five semi-structured expert 
interviews, this thesis uses an assemblage approach to examine the discourses surrounding 
Project Maven, the interplay of power between its elements, and their resulting 
configurations. It finds a tension between the compulsory powers of the 
Military-Technological Complex, exercised through structural, material relations, and the 
productive powers of Civil Society, exercised through the production and sanctioning of 
knowledge. It concludes that the development of consumer technologies for warfare and the 
associated emergence of a Military-Technological Complex reflects a broader unravelling of 


conventional ties between war, space and time. 
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7. Conclusion 


“Every gun that is made, every warship launched, every rocket fired signifies, in the 
final sense, a theft from those who hunger and are not fed, those who are cold and 
are not clothed. This world in arms is not spending money alone. It is spending the 
sweat of its laborers, the genius of its scientists, the hopes of its children.” 
Dwight D. Eisenhower, 1953 


1. Introduction 


“This is actually a bit more like building the atomic bomb than building Gmail,” claimed one 
Google worker about their contribution to the Algorithmic Warfare Cross-Functional Team, 
better known as Project Maven.’ Project Maven was a US Department of Defense (DoD) 
initiative that sought to use Artificial Intelligence (Al) technologies for automating drone 
footage analysis. In 2017, when the news broke that Google was involved, more than 4,000 
of its employees demanded that “Google should not be in the business of war,” presumably 
never anticipating that their code might be used for warfighting. The Campaign to Stop Killer 
Robots (a global coalition of NGOs) joined them in this call, expressing concern that this was 
the first step towards Lethal Autonomous Weapons Systems (LAWS).’? Although Google 
elected not to renew its Project Maven contract, its involvement nevertheless reflects a 
growing role for tech companies in an emerging “Military- Technological Complex.” 

Controversy surrounding Project Maven is directly relevant to the study of remote 
warfare, a phenomenon characterised by a shift away from ‘boots on the ground’ 
deployments towards light-footprint military interventions.* Technological innovation has 
played a critical role in this shift, enabling the US military to manage threats at low-cost and 
low-risk across the Middle East and Africa.* Replacing bodies with machines in war is feared 
by some to lead to Lethal Autonomous Weapons Systems (LAWS), or “killer robots.”° 
Understood as weapons capable of searching for, selecting and engaging targets 
autonomously (i.e. without human control), LAWS raise serious ethical, legal and political 
questions. Popular and academic preoccupation with their potential use in the future, 
however, risks overlooking their development in the present. 

As contemporary warfare depends on new technologies, the DoD is experiencing 
dramatic changes in their development and procurement. Al is at the centre of this 


transformation. Poised to initiate “the next industrial revolution,”’ the potential of Al in 


' Quoted in Tarnoff, B. (2019). Tech Workers Versus the Pentagon. Jacobin. Retrieved from 
https://jacobinmag.com/2018/06/google-project-maven-military-tech-workers 

? Wareham, M. (2018). Campaign to Stop Killer Robots Letter to Heads of Google and Alphabet. 
Retrieved from 
https://www.stopkillerrobots.org/wp-content/uploads/2018/04/KRC_LtrGoogle_12March2018.pdf 
3 Demmers, J., & Gould, L. (2020). The Remote Warfare Paradox. In Remote Warfare: 
Interdisciplinary Perspectives. E-International Relations. 

4 Ibid. 

5 Chamayou, G. (2016). A Theory of the Drone. The New Press. Chapter 23 

8 Scharre, P. (2018). Army of None: Autonomous Weapons and the Future of War. W. W. Norton. 180 
T Ibid. 18 


defence has been likened to gunpowder and the atomic bomb.’ Notably, however, the 
cutting-edge Al technologies sought by the DoD are not in government-funded research labs 
or among traditional defence contractors, but in private technology companies (e.g. Google). 
Studying technology and its development has long posed a challenge to scholars of IR, but 
there is a clear need to do so here.° Especially when “solutionism” increasingly, and perhaps 
mistakenly, offers technological solutions to political problems."° 

Project Maven was a staging ground in a larger battle over Al in defence. Drawing 
together various elements, including social actors (the DoD, tech companies, civil society 
campaigners), objectives (profit, safety, ethics), and discourses around a common issue, it 
revealed how their contestation and cooperation influences the development of Al in 
defence.'' Tracing power between these elements, via their social and material relations, 
provokes significant intrigue. Capturing this complexity is done here using the assemblage 
approach. With it, the heterogeneous elements converging around Project Maven seeking to 
govern the development of Al in defence can be delineated into a Military-Technological 
Complex and oppositional Civil Society Coalition, understood as two distinct but overlapping 
assemblages operating in the same field. 

Tech and defence elements coalescing around Project Maven indicates a porous 
civil-military divide. The concept of a Military-Technological Complex proposed here, 
understood as intensified cooperation between consumer technology companies and military 
actors, reflects a broader unravelling of conventional ties between war, space and time 
relevant to the study of remote warfare.” 

Drawing from and contributing to empirical debates on LAWS and remote warfare 
and theoretical debates on assemblage and technology in IR, this thesis remedies an 


important gap in the literature by answering the following question: 


How have practices of assemblage, as carried out by Google, the US Department of Defense, and the 
Campaign to Stop Killer Robots, formed a military-technological complex and oppositional civil society 


coalition seeking to govern the development of Al for defence from 2013 to 2020? 


8 PAX. (2019). Slippery Slope: The Arms Industry and Increasingly Autonomous Weapons. 
https://doi.org/10.7748/ns.26.22.28.s35 4 

° Leese, M., & Hoijtink, M. (2019). How (not) to talk about technology: International Relations and the 
question of agency. In Technology and Agency in International Relations. Routledge. 4 

1° Morozov, E. (2013). To Save Everything, Click Here: The Folly of Technological Solutionism. 
PublicAffairs. 

Li, T. M. (2007). Practices of assemblage and community forest management. Economy and 
Society, 36(2), 263-293. httos://doi.org/10.1080/03085140701254308 266 

12 Demmers, J., & Gould, L. (2018). An assemblage approach to liquid warfare: AFRICOM and the 
‘hunt’ for Joseph Kony. Security Dialogue, 49(5), 364-381. 


https://doi.org/10.1177/0967010618777890 366 


This thesis proceeds with an examination of the relevant existing empirical and theoretical 
debates in chapter two, followed by an outline of the methods used to examine this particular 
case in chapter three. Chapter four, forging alignments and rendering technical, explores 
how the elements of Project Maven assembled. Chapter five, authorising knowledge and 
managing failures, examines the legitimisation and contestation of knowledge between 
them. Chapter six, antipolitics and reassembly, considers the current status of the 


Military- Technological Complex. 


2. Theory 


“We can often learn as much from how and why questions have been asked in other 
circumstances as we can from the historical answers they yielded. Other people’s work is a 
spring-well of ideas about how to approach the world, even if it is about something we do not 


work on ourselves.” 


The case of Google as a defence contractor, via Project Maven, lies at the intersection of 
several academic debates. Much ink has already been spilt on the question of Lethal 
Autonomous Weapons Systems (LAWS), a field closely linked to a growing body of literature 
on “remote warfare,” arguably the driving force behind the automation of warfighting. Recent 
years have also seen extensive debate among scholars of International Relations on how 
they can conceptualise the role of new technologies and their development. In this regard, 
the assemblage has emerged as an appropriate lens for capturing the complexity of such 


issues." 


2.1. Lethal Autonomous Weapons Systems and Al in 
Warfare 


Debate relating to Lethal Autonomous Weapons Systems (LAWS) is a mile wide but an inch 
deep - vast but as-yet unsubstantiated. Ample discussion surrounds the potential political, 
legal and moral repercussions of deploying LAWS but there is little theoretically-informed 
analysis of their development. Given the revolutionary potential of such technologies, there 
is an urgent need to rectify this imbalance. 

Political challenges posed by the deployment of LAWS on battlefields are numerous. 
Unlike nuclear weapons, LAWS would require no hard-to-obtain materials and be relatively 
cheap to develop, leading some to fear a global robot arms race.'* Even those contending 


that “arms race” framing risks escalating rivalries nevertheless agree on the need to manage 


13 Lund, C. (2014). Of What Is This A Case? Analytical Movements in Qualitative Social Science 
Research. Human Organization, 73(3). 230 

4 Bousquet, A. (2013). Welcome to the Machine: Rethinking Technology and Society through 
Assemblage Theory. In M. Acuto & S. Curtis (Eds.), Reassembling International Theory: Assemblage 
Thinking and International Relations (p. 91+95). Palgrave Pivot. 

1S PAX for Peace. (2019). Don’t Be Evil: A Survey of the Tech Sector’s Stance on Lethal Autonomous 
Weapons. 9 
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the proliferation of these technologies.‘ In particular, the risk of LAWS proliferating among 
non-state actors seeking to “level the playing field” is severe.'’ Even among democratic 
states, LAWS would overcome the “bodybag’” problem and thus lower the threshold for going 
to war.'® In such a scenario, however, the obvious attraction of using LAWS - a ‘bloodless’ 
war between machines - is misguided. A change of footsoldiers does not alter the 
fundamental objective of war: “imposing and enduring costs to reveal relative capability or 
resolve.”’? In the absence of human soldiers to impose costs upon, belligerents may seek 
alternative means to demonstrate their resolve, e.g. by targeting non-military objectives and 
civilians.”° Indeed, as drones have enabled the removal of soldiers from battlefields, the 
inability for enemies to directly reciprocate violence has instigated fears of “blowback” by 
other means.”' 

Political concerns aside, the legal questions raised by LAWS are momentous. 
Formal and informal rules have guided conduct in warfare for as long as it has been carried 
out.”” Over time, International Humanitarian Law (IHL) has steadily evolved to keep up with 
the development of new methods of warfighting, as evidenced by bans on the use of 
Chemical and Biological Weapons, anti-personnel landmines and blinding laser weapons. 
Throughout history, however, IHL has been developed under the assumption that human 
intelligence is guiding conduct in combat, not artificial. As existing treaties make no 
reference to LAWS, it is unclear if they can be banned under the current legal framework.”° 
Nevertheless, because it is impossible to confidently predict the outcomes of commands on 
complex Al systems, there is a possibility of LAWS running afoul of IHL.“ Can a robot be 


entrusted with the ability to discriminate between legitimate targets and civilians? If it fails, 


18 Roff, H. M. (2019). The frame problem: The Al “arms race" isn’t one. Bulletin of the Atomic 
Scientists, 75(3), 95-98. httos://doi.org/10.1080/00963402.2019.1604836 97 

7 Chertoff, P. (2018). Perils of Lethal Autonomous Weapons Systems Proliferation: Preventing 
Non-State Acquisition. Geneva Centre for Security Policy, (2) 

'8 Leveringhaus, A. (2016). Ethics and Autonomous Weapons. Palgrave Pivot. 
https://doi.org/10.1057/978-1-137-52361-7 14 

19 Gartzke, E. (2019). Blood and robots: How remotely piloted vehicles and related technologies affect 
the politics of violence. Journal of Strategic Studies, 00(00), 1-31. 
https://doi.org/10.1080/01402390.2019.1643329 26 

?° Ibid. 24 

21 Hudson, L., Owens, C. S., & Flannes, M. (2011). Drone Warfare: Blowback from the New American 
Way of War. Middle East Policy, 18(3), 122-132. 

22 International Committee for the Red Cross. (2004). What is International Humanitarian Law? In 
Complete International Law. https://doi.org/10.1093/he/9780199679072.003.0012 

3 Krishnan, A. (2016). The Legality of Autonomous Weapons. In Killer Robots: Legality and Ethicality 
of Autonomous Weapons. Ashgate. 101 

4 Lin, P., Bekey, G., & Abney, K. (2009). Robots in War: Issues of Risk and Ethics. Ethics and 
Robotics, 49-67. 55 
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who can be held accountable? Some fear LAWS subverting the Hague Convention if they 
can act autonomously but not assume responsibility for the consequences.”° 

The ethics of using LAWS are fraught. At first glance, there is obvious appeal in 
using robots to do the “dull, dirty and dangerous” work in war as in civilian life. A robot never 
gets tired, stressed or traumatised. Their dispassionate nature, immune to the intoxicating 
effects of adrenaline, may even reduce the risk of civilian harm in conflict.2° From this 
perspective, if LAWS lessens the danger to soldiers and civilians alike, some suggest 
utilising them is not only judicious but an ethical imperative.”’ 

Even if LAWs could perfectly adhere to IHL and reduce harm to both combatants 
and civilians, many question whether they should. As a simple matter of dignity, some insist 


8B 


that taking a human life should involve a human decision.”® “Delegating the decision to kill to 


algorithms,” according to Rosert and Sauer, “is inhumane and unacceptable under any 
circumstances.” A compelling argument, but potentially relativised by the fact that there are 
many different interpretations of human dignity and many ways it can be violated - 
technological or otherwise.*° 

The political, ethical and legal concerns raised are troubling but overwhelmingly 
speculative. Are we jumping the (autonomous) gun? Such preoccupation with the future use 
of killer robots may obscure meaningful debate about their present day development. A few 
notable contributions from Lewis,*’ Taylor? and Verbruggen*® are to be commended for 
grappling with the development of these systems, but this remains an under-theorised field. 


Presumably owing to its novelty, Project Maven has thus far evaded serious academic 


6 Krishnan, A. (2016). The Legality of Autonomous Weapons. 106 

6 Lin, P., Bekey, G., & Abney, K. (2009). Robots in War. 52 

27 Arkin, R. C. (2009). Ethical robots in warfare. IEEE Technology and Society Magazine, 28(1), 
30-33. https://doi.org/10.1109/MTS.2009.931858; 

Foust, J. (2013, May 15). A Liberal Case for Drones. Foreign Policy. Retrieved from 
https://foreignpolicy.com/2013/05/15/a-liberal-case-for-drones/ 

28 Asaro, P. (2012). On banning autonomous weapon systems: Human rights, automation, and the 
dehumanization of lethal decision-making. International Review of the Red Cross, 94(886), 687-709. 
https://doi.org/10.1017/S1816383112000768 696 

° Rosert, E., & Sauer, F. (2019). Prohibiting Autonomous Weapons: Put Human Dignity First. Global 
Policy, 10(3), 370-375. https://doi.org/10.1111/1758-5899.12691 372 

3° Sharkey, A. (2019). Autonomous weapons systems, killer robots and human dignity. Ethics and 
Information Technology, 21(2), 75-87. https://doi.org/10.1007/s10676-018-9494-0 85 

3’ Lewis, L. (2019). Resolving the Battle over Artificial Intelligence in War. RUSI Journal, 164(5/6), 
62-71. https://doi.org/10. 1080/0307 1847.2019.1694228 

32 Taylor, T. (2019). Artificial Intelligence in Defence: When Al Meets Defence Acquisition Processes 
and Behaviours. RUSI Journal, 164(5-6), 72-81. https://doi.org/10.1080/03071847.2019.1694229 

33 Verbruggen, M. (2019). The Role of Civilian Innovation in the Development of Lethal Autonomous 
Weapon Systems. Global Policy, 10(3), 338-342. https://doi.org/10.1111/1758-5899.12663 
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scrutiny. One notable exception comes from Suchman,™ however it nevertheless engages 
primarily with the deployment rather than development of this technology. 

Common to these conjectural, dystopian perspectives on the future of warfare is the 
assumption that if LAWS technology exists, it will be used. Not a dramatic leap of faith but 
nevertheless it seems prudent to question what is driving the demand for these technologies 
in this domain; thereby drawing the line between where we stand and where some fear we 
are heading. LAWS will only be deployed, presumably, if there is some strategic, operational 
and/or tactical need for them. Understanding this will elucidate a fuller picture of why and 
how they are being developed. To satisfy this curiosity, we turn to the growing field of 


scholarship on remote warfare. 


2.2. Remote Warfare 


The best indication of warfare’s near future can be found in its recent past.*° With this in 
mind, LAWS can be contextualised by the ascendancy of “remote warfare.” Defined as “a 
strategy of countering threats at a distance, without the deployment of large military forces,” 
remote warfare takes a variety of forms, including the use of technological (e.g. drones) or 
physical (e.g. partner forces, private military contractors) proxies.*° By considering remote 
warfare from strategic-political and tactical-technological perspectives, LAWS can be 
considered a natural continuation of technologically-enabled warfighting among risk-averse 
Western states. 

The technologies of remote warfare - particularly drones - cannot be fully understood 
without attending to the strategic and political impetus for their use. “Stand-off’ warfighting 
runs deeper than any single technology. At the turn of the millennium, Ignatieff's analysis of 
the “Virtual War’ in Kosovo noted that Precision-Guided Munitions (PGMs) promised 


“speedy, risk-free victory.”*’ Contemporaneously, others noted the ascendancy of of “riskless 


34 Suchman, L. (2020). Algorithmic warfare and the reinvention of accuracy. Critical Studies on 
Security, 00(00), 1-13. https://doi.org/10.1080/21624887 .2020.1760587 

35 Harter, F., & Whittell, G. (2020, February). Kalashnikovs of Tomorrow. Tortoise. Retrieved from 
https://members.tortoisemedia.com/2020/02/03/kalashnikovs-of-tomorrow-ai-drones/content.html 

3° Watts, T. & Biegon, R. (2017). Defining Remote Warfare: Security Cooperation. In Remote Control. 
Retrieved from 
https://www.oxfordresearchgroup.org.uk/pages/category/conceptual-series-defining-remote-warfare 1 
37 Quoted in Sandergd, S. (2009). The Use of Air Power Today: Have New Ethical Challenges 
Occurred? In J. Hayward (Ed.), Air Power, Insurgency, and the “War on Terror.” 
https://doi.org/10.4324/9780429030772-13 227 
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warfare””’ and “risk-transfer militarism’*~ establishing a trend towards the asymmetrical 
deployment of force in Western Military operations.*° 

Patterns of risk-aversion among Western forces observed at the turn of the 
millennium have since accelerated. A combination of past failures, budgetary constraints, 
and increased political and popular scrutiny mean Western militaries exercise even greater 
caution in their troop deployments today.*’ Risk-aversion has not precluded military 
engagement entirely, but has prompted states to consider alternative means of projecting 
influence without committing large ground forces.” In his conceptualisation of “vicarious 
warfare,” Waldman therefore notes the pre-eminence of delegating force (to proxies and 
partner forces) and danger-proofing personnel (through airstrikes and remotely-piloted 
weapons systems) in contemporary military interventions.*? Delegation and danger-proofing 
respectively constitute the social and material forms of mitigating risk to US troops. Similar 
dynamics have been observed among other NATO members.** Put simply, the strategic 
appeal of remote warfare is perceived as efficiency: risking less and achieving more. 

From a tactical perspective, the efficiency engendered by remote warfare was made 
possible by technological innovation. For this reason, a burgeoning field of scholarship has 
emerged around the technologies of remote warfare, particularly drones - “the most visible 
application of the information age to contemporary warfare.”*° Chamayou’s seminal treatise 
on Drone Theory takes an expansive look at the political, ethical and legal implications of 
drone warfare, concluding that “remote war is a war of human machines against the human 
body... one side loses people; the other side loses toys.”“° While this analysis does consider 
the human impact of this technology, it remains overwhelmingly focused on the drone itself. 


Some argue this “fetishisation” of drone technologies (predominantly by military actors but 


38 Kahn, P. W. (2002). The Paradox of Riskless Warfare The Paradox of Riskless Warfare. Philosophy 
and Public Policy Quarterly, 22(3), 1-8. 

3° Shaw, M. (2002). Risk-Transfer Militarism, Small Massacres and the Historic Legitimacy of War. 
International Relations, 16(3), 343-359. 

“° Krieg, A. (2016). Externalizing the burden of war: The Obama Doctrine and US foreign policy in the 
Middle East. International Affairs, 92(1), 97-113. https://doi.org/10.1111/1468-2346.12506 101 

41 Knowles, E., & Watson, A. (2017). All Quiet on the Isis Front? Oxford Research Group Retrieved 
from http://remotecontrolproject.org 3 

“2 Heng, Y. K. (2018). The continuing resonance of the war as risk management perspective for 
understanding military interventions. Contemporary Security Policy, 39(4), 544-558. 
https://doi.org/10.1080/13523260.2018.1494670 549 

48 Waldman, T. (2018). Vicarious warfare: The counterproductive consequences of modern American 
military practice. Contemporary Security Policy, 39(2), 181-205. 
https://doi.org/10.1080/13523260.2017.1393201 189 

“4 Knowles, E., & Abigail Watson. (2018). Remote Warfare: lessons learned from contemporary 
theatres. Oxford Research Group. 8 

45 Horowitz, M. C. (2020). Do Emerging Military Technologies Matter for International Politics? Annual 
Review of Political Science, 23, 385-400. https://doi.org/10.1146/annurev-polisci-0507 18-032725387 
46 Chamayou, G. (2016). Drone Theory. The New Press. Epilogue 
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also in academia) mystifies their human relations.*” Myopic focus on technologies alone thus 
risks presenting a false image of “war without bodies.”“® 

Scholars of remote warfare are not alone in their tendency to perceive technology 
deterministically. Despite the centrality of technology to IR, its study too-often assumes it is 
either entirely controlled by humans or distinctly separate from human agency.” In line with 
Latour’s concept of the Black Box, “technical work is made invisible by its own success,” 
leaving us unaware of its broader significance.®*° As Flusser quipped, however, “technology 
has become too serious a matter to be left to technicians.”*' There is thus a need to 
apprehend the complex socio-technical systems that produce and are served by 
technologies. Or, as Hoijtink and Leese invite, to “render them political” and consider “the 
politics that go into technology, as well as the politics that emanate from technology.” In 
such socio-technical systems, accounting for the interaction of humans, technologies and 
structures invites analysis of how agency and power may be produced, distributed and 


transformed.°° 


2.3. Assemblage 


Situated at the intersection of multiple fields of academic study, political debate and 
technological innovation, the LAWS development can be approached from several angles. 
Project Maven, as a complex and mutable amalgamation of elements - Google, the DoD, the 
Campaign to Stop Killer Robots, and myriad others - converging around the development of 
Al in defence, demands a holistic approach. For this, we turn to the assemblage. 

Originally articulated by Deleuze and Guattari, the assemblage is an ontological 
framework denoting both the act of arranging heterogeneous elements in a complex social 
system and the resulting arrangement itself. It can therefore be considered both a process 
and a product, a simplification necessitated by a lack of conceptual clarity in the works of 
Deleuze and Guattari, who offer half a dozen different definitions.°° DeLanda nevertheless 


helpfully weaves a poetic, albeit incomplete, interpretation: 


47 Ronald Shaw, I. G., & Akhter, M. (2012). The Unbearable Humanness of Drone Warfare in FATA, 
Pakistan. Antipode, 44(4), 1490-1509. https://doi.org/10.1111/j.1467-8330.2011.00940.x 1501 

“8 Ibid. 

4° Leese, M., & Hoijtink, M. (2019). How (not) to talk about technology. 10 

5° Latour, B. (1999). Pandora’s Hope: Essays on the Reality of Science Studies. Harvard University 
Press https://doi.org/10.1017/CBO9781107415324.004 304 

5’ Flusser, V. (2011). Into the Universe of Technical Images. University of Minnesota Press. 65 

52 Leese, M., & Hoijtink, M. (2019). How (not) to talk about technology. 3 

53 Ibid. 

54 Bogue, R. (1989). Deleuze and Guattari. London and New York: Routledge. 174 

55> DeLanda, M. (2016). Assemblage Theory. https://doi.org/10.5040/9781350096769.0003 1 
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“A multiplicity which is made up of many heterogeneous terms and which establishes liaisons, 
relations between them... Thus, the assemblage’s only unity is that of a co-functioning: it is a 
symbiosis, a ‘sympathy’. It is never filiations which are important, but alliances, alloys; these 


are not successions, lines of descent, but contagions, epidemics, the wind.”°6 


Such a metaphysical conceptualisation is thought-provoking but of limited practical value. 
“Heterogeneous terms establishing liaisons” appear to fit the bill for Project Maven, itself a 
“co-functioning symbiosis” of public and private actors, but how can this framework be 
applied? Fortunately, a rich intellectual tradition of scholarship operationalising the 
assemblage for studying social phenomena is at hand. From this, two main issues must be 
considered: the process of assembling (i.e. its elements and their relations shaping it) and 
the resulting outcome (i.e. what it seeks to actually do). 

A practical definition of assemblages comes from Abrahamsen and Williams, as 
“structures and networks in which a range of different actors and normativities interact, 
cooperate and compete to produce new institutions, practices and forms.”*’ For this 
research, the interaction of normativities appears particularly intriguing, as it speaks to the 
distributed and dissipated nature of agency within a body lacking central organisation. A 
similarly expansive interpretation of agency is seen by Bennett’s study of the electrical power 
grid, in which an assemblage includes not only human actors but technological, cultural and 
atmospheric elements, too.® 

Dittmer elaborates that this “posthuman turn” should be considered with respect to 
the relational ontology of the assemblage.®*. This means recognising the agency of wholes 
and parts, rather than either exclusively. So while the heterogeneous elements themselves 
provoke intrigue, power in the assemblage does not lie within them but in relations between 
them.®' Simply drawing these connections is insufficient, however, lest we undertake a 


“joining up exercise” rendering only thin description.” Thicker analysis is achieved through 


°6 Ibid. 

87 Abrahamsen, R., & Williams, M. (2011). Power and Governance: Global Assemblages and the 
Security Field. In Security beyond the state (pp. 89-121). 90 

8 Bennett, J. (2006). The agency of assemblages and the North American Blackout. Political 
Theologies: Public Religions in a Post-Secular World, 17(3), 602-616. 
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attending to the tensions, contradictions and clashes which, paradoxically, reveal how the 
assemblage actually takes form.® 

Studying the friction binding the assemblage invites analysis of its practices of in- and 
exclusion.“ With respect to this, it helps to draw together Barnett and Duvall’s 
conceptualisation of compulsory power (direct capacity to control others, through structural 
relations) and productive power (constitution of social actors capable of effective action, 


through diffuse social relations).°° 


Doing so bridges the coercion of material relations with the 
Foucauldian persuasion of social relations. At their intersection, “productive power makes 
some instances of compulsory power possible and legitimate. And, in turn,... compulsory 
power shapes the terms of meaning that influence how actors see what is possible and 
desirable.”* Power in this case is not centralised, or even distributed equally, but is “power 
as plurality.”°’ In an assemblage such as Project Maven, featuring the complex interaction of 
economic incentives, ethical arguments and political norms, this holistic conceptualisation of 
power is highly appropriate. 

Mindful of the distribution of agency and primacy of relations to assembling, we turn 
to the resulting arrangement. Critically, although the assemblage is an “arrangement,” it 
should not be interpreted as a specific body, but a means of understanding how 


68 « 


heterogeneous elements can coalesce without actually forming a coherent whole.” “Unity 


across difference” in this manner gives rise to unique forms of stability and change, order 
and disruption.®° Although this suggests the assemblage is a transient phenomenon, some 
contend they may be durable if rooted in historical connections.”” Temporally and spatially, 
then, the form of assemblages is fluid or at least liable to change.”' 

If change and difference are so central to the assemblage, what brings order to the 
chaos? Elements of an assemblage must, presumably, coalesce around some point of 


convergence. Anderson and McFarlane assert that an assemblage is productive of certain 
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effects.”* In the case of Abrahamsen and Williams, this is to influence what is regarded as 
“possible and desirable” by its elements.” Similarly, for Demmers and Gould, the “effect” of 
the assemblage is as a governance formation.“ They follow Li in her understanding of the 
assemblage as unified by “the will to govern, or... the will to improve: the attempt to direct 
conduct and intervene in social processes to produce desired outcomes and avert undesired 
ones.””° For this research, this definition can be augmented by Dafoe’s conceptualisation of 
Al governance as, “how humanity can best navigate the transition to advanced Al systems.” 
’® This leaves us well-attuned to what pulls the actors in Project Maven together: the attempt 
to direct the transition to advanced Al systems to produce desired outcomes or avert 
undesired ones. 

For using the assemblage as a heuristic tool, the most coherent and practicable 
operationalisation comes from Li.” In her study of community forest management, she 
identifies six practices of assemblage. First, forging alignments is the work of linking together 
the objectives of the elements of an assemblage by means of a shared problem definition.” 
Second, rendering technical involves reducing the social world to a formulaic “problem (a) 
plus intervention (b) will produce (c), a beneficial result.” Third, authorising knowledge is the 
specification of body of knowledge, confirmation of enabling assumptions and containment 
of critique. Fourth, managing failures and contradictions means presenting failure as 
rectifiable, smoothing out contradictions and devising compromises. Fifth, antipolitics 
re-poses political questions as matters of technique, thus demarcating the acceptable 
boundaries of debate. Sixth, reassembly grafts new elements and re-works old ones in the 
assemblage. Collectively, these practices reveal how the elements of an assemblage cohere 
and act.” In the case of Project Maven, they articulate how its actors assemble into a 
Military- Technological Complex and antagonistic Civil Society Coalition, and how each seeks 


to influence the transition to advanced Al systems in the context of defence. 
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2.4. Research Gap 


This research contributes to all debates outlined above. Empirically-inspired by the 
burgeoning field of research on LAWS and a paucity of theoretically-informed research into 
their development, it links the near-future of “robot warfare” with existing practices of remote 
warfare. Using the analytical frame of assemblage to understand “power in complexity,”®° it 
contributes not only to the academic debate around assemblage but also wider discussions 
concerning power, agency and technology in international relations. Theorising this 
under-examined but critically important phenomena in the emergence of a 
Military- Technological Complex will be critically important to understanding the future of Al in 


war. This research gap is addressed with the following question: 


How have practices of assemblage, as carried out by Google, the US Department of Defense, and the 
Campaign to Stop Killer Robots, formed a military-technological complex and oppositional civil society 


coalition seeking to govern the development of Al for defence from 2013 to 2020? 


Using the practices of assemblage articulated by Li, the following sub-questions have been 
formulated to answer it: 

1. What shared problem definition(s) are produced by the actors involved? 

2. What technical solutions are proposed? 


3. Which new actors and discourses are included and excluded from the Military-Technological 
Complex and Civil Society Coalition, and how? 


4. What frictions or failures emerge and how are they managed? 
5. How are political questions re-posed as matters of technique? 


6. What is the re-assembled form of the Military-Technological Complex and Civil Society 
Coalition? 
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3. Method 


“The term method in qualitative research is meant to imply more than a practical technique or 
procedure for gaining data. It also implies a data-generation and engagement process 


involving activities that are intellectual, analytical and interpretive.”®' 


Social research is an act of linking ideas and evidence to produce a representation of an 
aspect of social life.°* Connecting ideas and evidence is the analytical frame of assemblage 
outlined in the previous section. It is vital that evidence is collected and analysed in a 
manner consistent with the analytical frame. To this end, the research strategy will be 
detailed here with reference to its ontological and epistemological positioning. An 
explanation of the research methods used will follow, concluding with an appraisal of their 


limitations. 


3.1. Research Strategy 


A research strategy sets out the means of linking evidence to concepts and theory for 
answering the questions posed.*’ These means must be ontologically and epistemologically 
consistent with what they examine. Here the ontology and epistemology of the research will 
be explained, with reference to the analytical frame. 

The fundamental ontological divide in social science is between individualism, 
treating the individual as the elementary unit of social life, and structuralism, emphasising 
institutions beyond the control of individuals.®* While Li notes that the assemblage signifies 
agency insofar as it recognises the work undertaken by certain subjects, she equally 
recognises it as the product of specific, enabling circumstances.®° Furthermore, elements of 
an assemblage can be extremely varied, encompassing somatic, technological, cultural and 
atmospheric elements.® Traversing such an ontologically diverse landscape necessitates 
transcending the structure/agency debate. As a fluid arrangement of heterogeneous 
elements, defined by the interactions between them, it can be said to be ontologically holistic 


or, more accurately, relational.®” Such ontological ambiguity does not diminish its analytical 
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value, however, but allows the researcher “to attend to the agency of wholes and parts, not 
one or the other” (emphasis in original).°° 

The basic epistemological divide is between attempts at explaining and 
understanding the social world.2° The former is inherently positivist and the latter 
interpretivist. An interpretivist epistemology is better suited to the assemblage approach, 
used to understand the processes of how assemblages are formed rather than the causes of 
why they take shape. This is supported by previous applications of the assemblage 
investigating how different agents coalesce around the issue of forest management,°? how 


housing governance shapes citizenship,°" 


and how private security produces new forms of 
security governance.” 

A qualitative research strategy is consistent with the ontology and epistemology of 
the assemblage approach, as this research seeks to understand how different actors 
perceive and produce discourses concerning problems, solutions, knowledge and failures - 


none of which can be objectively quantified. 


3.2. Research Design 


Research design is the means of linking this evidence to ideas to answer the questions 
posed. This section therefore outlines the sampling and temporal focus of this research. 

As a global phenomenon, dissipated among all the elements of the assemblage in 
their respective offices, parliaments, barracks, publications and meetings, there is no clearly 
demarcated field where the “battle over Al” is taking place. Nevertheless, this research 
focuses on the case of Project Maven, an archetypal manifestation of the 
Military-Technological Complex and Civil Society Coalition. Carefully studying a single case 
in this manner strengthens internal validity.°° 

The units of analysis examined include Google, the United States Department of 
Defence, and the Campaign to Stop Killer Robots. These were purposively sampled for their 


involvement in the development of Al for military purposes. This is not intended to be 
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statistically representative as this phenomenon is studied for its social scientific significance, 
thus requiring highly selective sampling in order to generate useful data.“ 

The temporal focus of this research is from November 2014 to June 2020. Beginning 
with the US Secretary of Defence Chuck Hagel proposing the Third Offset Strategy, a plan 
for outmanoeuvring the United States’ adversaries through innovative uses of new 
technologies, ending in June 2020 to allow for data generation up to the contemporary status 
of this phenomenon. This six year period is long enough to observe changes and 
developments in the assemblages, particularly through the establishment, successes and 
failures of Project Maven. 

Data collection was split into four phases of (1) contextualisation; (2) 
problematisation; (3) contestation; and (4) resolution. The first phase (contextualisation) 
focused on mapping the actors involved and their interests and capacities. The second 
phase (problematisation) sought to answer sub-questions 1. and 2. by focusing on problem 
definitions and technical solutions articulated by the actors. The third phase (contestation) 
sought to answer sub-questions 3. and 4. by examining tensions and contradictions between 
the actors and how they were publicly articulated. Lastly, the fourth phase (resolution) 
sought to answer sub-questions 5. and 6. by outlining the current status of the elements in 


their “reassembled” state. 


3.3. Research Methods 


Aktouf defines method as, “the logical procedure employed by a science, i.e. the set of 
specific practices it uses to render the development of its demonstrations clear, 
understandable and irrefutable.” This section therefore outlines the data collection 
techniques used. 

This research relied predominantly on document analysis and qualitative interviewing 
for data generation. These methods are consistent with the holistic, relational ontology of the 
research and its desire to understand discourses, knowledge, and interactions between the 
units of analysis.°° 

With regards to the documents analysed, a number of key texts were chosen for 
Google (e.g. investor relations communications, blog posts, public statements); the DoD 


(e.g. speeches, strategies, directives and memorandums); and the Campaign to Stop Killer 
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Robots (e.g. public statements, speeches, pamphlets). A variety of journalistic sources from 
newspapers, magazines and other companies involved in Project Maven provided contextual 
data. 

Document analysis was supplemented by five semi-structured interviews conducted 
with four respondents. These triangulated the data generated from document analysis and 
complemented the perspectives of official sources with individual perspectives from those 
intimately familiar with the phenomenon. To this end, two tech workers-turned-activists who 
had worked on Project Maven and two members of the Campaign steering committee were 
interviewed. 

Data generated from documents and interviews alike were coded in Nvivo 12 to six 


themes corresponding to the practices of assemblage identified by Li.%” 


3.4. Limitations 


A combination of methodological limitations and external constraints inhibited the course of 
this research. Here their impact and attempts at mitigating them will be outlined. 

Owing to the sensitivity of the phenomenon investigated, this research depended on 
publicly-available information, which was more readily-available from civil society actors. 
Although two of the respondents interviewed were tech workers from companies involved in 
Project Maven, their status as vocal critics of it meant they were closer aligned with the other 
two respondents, members of the Campaign steering committee. A combination of 
inaccessibility and constraints on time and resources precluded interviews with the 
‘proponents’ of the Military-Technological Complex, forcing a _ reliance on_ public 
communications, e.g. speeches and directives. Further research on the _ individual 
perspectives of the military actors involved would undoubtedly enhance understanding of 
this phenomenon. 

It would be remiss not to mention the effect of the global covid-19 pandemic on the 
course of this research. Plans to conduct ethnographic fieldwork at a number of events 
(conferences and arms fairs) were unfortunately cancelled. Fortunately, however, video-call 
technology enabled me to interview respondents in multiple countries. Although these lacked 


much of the nuance afforded by face-to-face interviews, they were nevertheless valuable. 
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4. Forging Alignments & Rendering Technical 


“Forging alignments: the will to govern as a point of convergence and fracture.’®° 


“Rendering technical: framing the arena of intervention.’®° 


Assemblages, for all their heterogeneity, require a point of convergence to coalesce around. 
Forging alignments in this way is “the work of linking together the objectives of the various 
parties to an assemblage,” by a joint problem definition."*' With this shared perception of a 
problem, the elements of an assemblage begin rendering technical, reducing it to “a diagram 
in which problem (a) plus intervention (b) will produce (c), a beneficial result.”'© 

This chapter therefore explores how these elements coalesced around the issue of 
Al in defence and proposed an intervention. Frictions between the two configurations were 
immediate and clear, reinforcing the notion that “the assemblage is far from seamless.”'™ 
This chapter will begin with an exploration of how the military-tech and civil society elements 
define the problem, followed by their proposed solutions, concluding with a comparison of 
the two which will set the stage for the subsequent chapter, Authorising Knowledge and 


Managing Failures. 


4.1. Problem Definition 


“U.S. troops should not be sent into fair fights,” according to Joe Dunford, Chairman of the 
Joint Chiefs of Staff." To ensure ‘unfair’ fights, the Department of Defence has long sought 
the latest and greatest defence technologies, traditionally, researched and developed by a 
dedicated and well-established “Military-Industrial Complex.” Recently, however, as digital 
innovations have begun to outrank the physical, the DoD has turned to consumer technology 
companies (such as Google) to fulfil their needs. Forging new partnerships with these 


companies thus presented a technical problem. 
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For civil society actors involved in humanitarian disarmament, the application of Al for 
defence posed a serious threat. So while the DoD forged partnerships with industry to 
channel Al innovation towards its own ends, particularly with Project Maven, civil society 
sought to influence the same process. By contrast, however, civil society regarded this issue 


and its solutions in political rather than technical terms. 


4.1.1. Military-Technological Complex 

“To overcome challenges to our military superiority,” declared the US Secretary of Defense 
Chuck Hagel in 2014, “we must change the way we innovate, operate, and do business.”'®> 
Noticing tides of change lapping at his feet, Hagel recognised that new ways of war required 
new ways of business. From his perspective, the problem of Al in defence was one of 
access: the technology existed, but was out of reach of the DoD. Cooperation with new 
industrial partners would be essential to developing these new technologies and, in turn, 
maintaining American military superiority. 

To understand why the Department of Defense needed to forge new alignments with 
industry, it helps to consider previous practices of military research and development (R&D) 
and their (un)suitability for purpose. In the post-WWII era, a “Military Industrial Complex” 
emerged among the civilian industrial conglomerates (e.g. Boeing and General Motors).'°° 
Simultaneously, significant public investments into military R&D boomed and led to several 
technologies now commonplace in the civilian sphere, such as GPS."” In fact, this 
cross-pollination of public funding and technological innovation between the military and 
civilian spheres transformed Silicon Valley’s from a “landscape of fruit orchards into a hub of 
electronics production and innovations.”'® 

At the conclusion of the Cold War, the Military-Industrial Complex lost its raison 
d’etre.'®° Fifty-five billion dollars of acquisitions and mergers followed, as many engineering 
companies parted with their defence businesses, consolidating the defence industry into five 
“titans.”""° Concurrently, DoD procurement practices became cumbersome and convoluted, 


resulting in $46bn wasted between 2001 and 2011 on weapons systems that never entered 
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production.''’ Most acquisition programmes took years or even decades to reach the 
battlefield."’* Tightening belts and a slow production line blunted the technical edge of the 
Military-Industrial Complex. Today, most R&D investments are dominated by the private 
sector.''® 

By 2014, when Hagel announced the Third Offset Strategy, it was clear that digital 
technologies were the future of defence. As the leader of Project Maven Jack Shanahan 
noted, “the future battlespace is constructed of not only ships, tanks, missiles, and satellites, 
but also algorithms, networks, and sensor grids.”''* Hagel obliquely referred to LAWS with 
his warning that the US army “could one day go into battle confronting a range of advanced 
technologies that limit our freedom of maneuver.”"*® Deputy Secretary of Defense Bob Work, 
by contrast, was more explicit: “10 years from now if the first person through a breach isn't a 
fricking robot, shame on us”'’® Moreover, the official DoD directive on LAWS declared its 
intention to develop autonomous weapons with “appropriate levels of human judgment” that 
would satisfy “the law of war, applicable treaties, weapon system safety rules, and applicable 
rules of engagement.”'” From this, it appears the problem was perceived not in the 
technology itself but in developing it to a certain standard. It could not do so alone. 

Technological innovation took centre stage in the Third Offset Strategy but would 


require new actors, by Hagel’s own admission: 


“[The] DoD no longer has exclusive access to the most cutting-edge technology or the ability 
to spur or control the development of new technologies the way we once did. So we will 
actively seek proposals from the private sector... those firms and academic institutions 


outside DoD’s traditional orbit.”""® 


Technology companies, by contrast, were not as public in their intentions of working with the 


DoD. Despite Silicon Valley’s government-funded roots, its radically different working culture 


™ Weisgerber, M. (2014, November). Slow and Steady is Losing the Defense Acquisition Race. 
Government Executive. Retrieved from 
https://www.govexec.com/feature/slow-and-steady-losing-defense-acquisition-race/ 

"2 Allen, G. C. (2017, December). Project Maven brings Al to the fight against ISIS. Bulletin of the 
Atomic Scientists. Retrieved from 


https://thebulletin.org/2017/12/project-maven-brings-ai-to-the-fight-against-isis/ 
"3 Lewis, L. (2019). Resolving the Battle over Artificial Intelligence in War. RUSI Journal, 164(5/6), 


62-71. https://doi.org/10.1080/0307 1847.2019.1694228 63 

™4 Weinbaum, B. C., & Shanahan, J. N. T. (2018). 35.- Intelligence in a Data-Driven Age. Joint Forces 
Quarterly, 90(3rd Quarter), 4-9. 5 

8 Hagel, C. (2014). Reagan National Defense Forum Keynote. 

18 Work, B. (2015). Reagan Defense Forum: The Third Offset Strategy. Retrieved from 
https:/Awww.defense.gov/Newsroom/Speeches/Speech/Article/628246/reagan-defense-forum-the-thir 
d-offset-strategy/ 

7 US Department of Defense. (2012). Directive 3000.09 (pp. 1-15). Retrieved from 


https:/Awww.esd.whs.mil/portals/54/documents/dd/issuances/dodd/300009p.pdf 3 
"8 Hagel, C. (2014). Reagan National Defense Forum Keynote. 


26 


of “disruption” and willingness to “fail fast, fail often” were institutionally alien to the DoD.” 
Beyond these normative issues, many technology companies regarded the Pentagon as too 
small a customer to work with, especially given the bureaucracy involved.'*° Similarly, the 
huge sets of training data required to make useful machine-learning algorithms are hard to 
come by for defence purposes. '*' Notably, well before Project Maven was even established, 
Google was making significant inroads in robotics development but appeared unwilling to 
work with the Pentagon, however it was unclear whether this was driven by ideological or 
market considerations.’ Nevertheless, it was clear that for many tech companies the 
problem was perceived in a technical manner similar to the DoD: the issue was not 


necessarily developing the technology but establishing the partnerships. 


4.1.2. Civil Society Coalition 
The issue of “robot arms control” was familiar to civil society well before Project Maven’s 
formation in 2017. The Human Rights Watch (HRW) arms division was studying the issue of 
LAWS as early as 2010, in conjunction with civil society groups conducting research on the 


123 It was 


issue of armed drones - a clear sign of its relevance to scholars of remote warfare. 
not long until HRW was approached by a group of roboticists who had established the 
International Committee for Robot Arms Control (ICRAC) in 2009. Billed as a “not-for-profit 
association committed to the peaceful use of robotics in the service of humanity and the 
regulation of robot weapons,” they were technically knowledgeable but lacked the expertise, 
experience and networks in international diplomacy HRW held.‘ 

In October 2012, HRW assembled a small team of other non-governmental 
organisations concerned about removing human control from the use of force to co-found 
The Campaign To Stop Killer Robots (“The Campaign” hereafter). At this embryonic stage, 
the coalition was born of both necessity and strategy. In Campaign Coordinator Mary 


Wareham’s words: 


“When we started to look at this concern, we realised that we [HRW] were going to have to 
cooperate with other non-governmental organisations to have a sustained civil society 


campaign to get a new treaty... We’re a large NGO, but the arms division is a small part of 
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Human Rights Watch and our impact is multiplied by working with other, like minded 


non-governmental organisations.”'° 


Fellow member of the Campaign steering committee, Frank Slijper of PAX, elaborated that 
coalitions are “more the rule than the exception” when working on campaigns of this sort; 
“you’re much more effective if you’re one voice towards the international community.”'*° The 
forging of alignments around a perceived political problem at this nascent stage thus 
appears an organic process. 

Those attending the first official meeting of the Campaign were unified by “a shared 
concern about fully autonomous weapons,” according to Laura Boillot of Article 36.'*” An 
all-star lineup of more than thirty civil society organisations attended this inaugural meeting. 
As well as HRW, ICRAC, and Article 36, fellow human rights heavyweights such as Amnesty 
International, PAX and the Nobel Women’s Initiative entered the fray.'*® The presence of 
Drone Wars UK and Mines Action Canada similarly evidence the relevance of this issue to 
wider questions of remote warfare and arms control. The “shared concern’ Boillot alluded to 
was apparently self-evident to those in attendance. In her opening remarks, Wareham 
questioned, “I was meant to talk about why we want to campaign to stop killer robots, but I’d 
like to switch the question and ask why anyone would not want to campaign to ban killer 
robots?”'*? These members of the assemblage were evidently unified by a belief in the 
fundamentally wrong nature of LAWS (the “ugh” factor, as they put it).'*° Nevertheless, to 
win over wider support they articulated the problem in legal, ethical and moral terms. 

Legal concerns played a central role in the Campaign from the start. LAWS, the 
Campaign argued, may “lack the human judgment necessary” to adhere to IHL and human 
rights law, and an “accountability gap” leaves ambiguities over who would be legally 
responsible for their actions.'*' That said, this research found no evidence of any tech 
workers legitimately concerned their work would render them legally accountable for war 
crimes. 

More prominent in the Campaign’s messaging were ethical concerns, which also held 


far greater purchase with tech workers. LAWS cross a “moral threshold,” according to the 
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campaign, and as non-human actors they lack “the inherently human characteristics such as 
compassion that are necessary to make complex ethical choices.”'** Philosopher Peter 
Asaro rebuked the ethical argument that the “perfect” killer robot may minimise civilian harm 
by asserting the likelinood of a robot being used in this way is far lower than the probability 
of dangerous systems proliferating. '*° 

Inherent to the concerns outlined above was the perception of a political rather than 
technical problem. Understandably, given the nature of the organisations involved at this 
early stage, discussions focused on the political elements to be overcome. Interestingly, 
however, “people working in science and technology” (i.e. tech workers) were identified as a 
potential constituency opposing a ban."™ In actuality, many of these tech workers shared the 
Campaign’s concerns. A 2015 open letter which has now been signed by more than 4500 
roboticists and tech workers asserted that “just as most chemists and biologists have no 
interest in building chemical or biological weapons, most Al researchers have no interest in 
building Al weapons.”'*° Connections between the tech community and nongovernmental 
organisations were therefore not formally or deliberately drawn, but were nevertheless 


beginning to assemble. 


4.2. Rendering Technical 


4.2.1. Military-Technological Complex 
From the Third Offset Strategy, Al technologies were clearly considered imperative to 
maintaining American military strength. Incapable of mandating cooperation between private 
industry and the military, however, the DoD had to rely on persuasion rather than coercion to 
acquire the technologies it sought.'®° Accordingly, the “solution” for the DoD appeared to be 
becoming better in building relationships with contractors. And so it began courting tech 


companies with new contracting processes. 
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In April 2015, then-Secretary of Defense Ash Carter delivered a speech at Stanford 
University - the first visit to Silicon Valley from a Secretary of Defense in twenty years.'” 
Centred around innovation, the speech culminated in the announcement of the Defense 
Innovation Unit Experimental (DIUx). This new DoD body, located in Silicon Valley (“a nexus 
of innovation”), would work to “strengthen existing relationships and build new ones; help 
scout for new technologies; and help function as a local interface for the department.”'*® 

In addition to the DIUx, the Defense Innovation Advisory Board (DIB) was 
established the following year. This collection of Silicon Valley luminaries, chaired by Eric 
Schmidt (formerly of Alphabet Inc., the parent company to Google), were assembled to 
provide the Pentagon access to “the brightest technical minds focused on innovation.”'*° 
Drawing industry voices like Schmidt into the assemblage suggested that with the right 
expertise, the challenges of developing Al for defence could be overcome. The role of the 
DIB in legitimising the DoD’s endeavours in this field will be elaborated in the following 
chapter: authorising knowledge & managing failures. 

Forging alignments with tech companies accelerated dramatically in 2017 with the 
creation of the Algorithmic Warfare Cross Functional Team (AWCFT). Better known as 
Project Maven, the AWCFT sought “to accelerate DoD's integration of big data and machine 
learning... to tum the enormous volume of data available to DoD into actionable intelligence 
and insights at speed”"*° Specifically, its goal of automating the “Processing, Exploitation and 
Dissemination [PED] for Tactical Unmanned Aerial Systems” meant using Machine Learning 
to automatically analyse drone footage for objects of interest, freeing up human analysts for 
higher-level work."*' 

It is important to note that those leading Project Maven in the DoD were not selected 
on the basis of technological expertise but their skill in building relationships. "*? Although the 


role of Google was not public at this early stage, a public “industry day” hosted by the DoD in 
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October 2017 was attended by more than 300 industry and academic partners - a clear sign 
of DoD enthusiasm for forging alignments with the tech industry. ° 

Beyond the concrete goals of Maven, it had an important underlying purpose as a 
“pathfinder” mission, the “spark that kindles the flame front of artificial intelligence across the 
rest of the [Defense] Department,” according to its leader Lt. Gen. Jack Shanahan.” Its 
founding memo emphasised agility: “after successful sprints in support of Intelligence, 
Surveillance, Reconnaissance (ISR) PED, the AWCFT will prioritize the integration of similar 


technologies into other defense intelligence mission areas”'*° 


(emphasis added). The 
Pentagon was thus framing the solution as being to move faster and work closer with 
industry in adopting and integrating algorithmic technologies. 

For the DoD, then, just as the “problem” of integrating Al technologies was a 
logistical one, the proposed solution was equally practical. Initiatives like the DIUx, DIB and 
Project Maven indicated that with enough energy and skill, it could forge the necessary 
alignments with tech companies to acquire these technologies. Elements gathering around 
the problem of Al in defence from this perspective were thus already crystallising as its 


solution - closer cooperation. 


4.2.2. Civil Society Coalition 
It follows from a shared belief in the political nature of the problem among civil society that 
the solution would be a political endeavour. Civil society was interested in rendering the 
issue political rather than technical, raising awareness of what it considered inherently 
unsolvable issues with Al in defence. 

At the Campaign launch in 2013, its demand was explicit: “a pre-emptive and 
comprehensive ban on the development, production, and use of fully autonomous 
weapons... through an international treaty, as well as through national laws and other 
measures.”"“° Notably, its call does not specifically target Al or even Al-enabled weaponry 
but simply autonomous weapons. Al is understandably what many fear to enable the 


“cognitisation” of machines, i.e. making them faster and smarter than humans at certain 
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tasks.'*” With respect to this, the solution articulated by the Campaign is thus not so much 


technical but political. Campaign coordinator Mary Wareham elaborated: 


“We're not just talking about a single weapon but a whole way of warfighting. And we’re 
talking about systems, not just weapons, so that’s why the campaign call is both framed ina 
negative sense and a positive sense... “Ban killer robots” is understood by everybody, but the 
softer, more nuanced approach is to say we are working to retain meaningful human control 
over the use of force and that's the part that involves any type of weapons system you can 


imagine.”'48 


Humanising the issue in this manner offers a powerful counterpoint to technical perceptions 
of this issue. In doing so, the Campaign can appeal to a broader constituency. As its Silicon 


Valley lead Marta Kosmyna told an audience of Silicon Valley tech workers: 


“The campaign is not anti-tech, we’re not anti-Al, we think robots are really cool. We just think 
that when you take them to that level where you have a system that's lethal... take a minute 
and pause and think through some of the unintended consequences that these weapons can 


have.”"49 


In rendering a political solution rather than a technical one, the Campaign was explicit in 
what it sought to ban and could therefore appeal to a broader base for support. While it was 
widely recognised that Al would enable automation in this context, its proposed solution 
avoided mentioning that specifically. Relative to the DoD, civil society was having a wider 
debate about the morality and ethics of killing, and what could be considered acceptable 


conduct in warfare. To resolve this, clear standards had to be set and enforced. 


4.3. Analysing Alignment 


Evidently, the elements of both assemblages perceived a problem. There was, however, a 
profound disparity in how it was perceived by the DoD and technology companies, on the 
one hand, and civil society, on the other. Their respective framing revealed the compulsory, 
material power of the DoD, seeking to issue sizable contracts for new technologies, and the 
productive, social power of civil society, seeking to highlight the ethical issues involved. 
Compulsory power in the Military-Technological Complex was influencing actors via DoD 


resources. Civil society productive power, by contrast, was defining “the (im)possible, the 
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(im)probable, the natural, the normal, what counts as a problem.”'*° Tension between the 
two helped bind the elements of these assemblages in their different configurations. 

Identifying the vast potential of Al technologies in defence, the DoD perceived a 
technical problem of byzantine contracting processes hindering their ability to procure it from 
tech companies. The technical solution to this technical problem was an agile and 
fast-moving process (in Project Maven), facilitated by bodies tasked with paving the way for 
deeper cooperation (the DIUx and DIB). For the DoD, it was convenient to emphasise the 
technical elements of this issue and sideline the numerous legal, ethical and political issues. 
By framing this as a technical question of unlocking the potential of the private sector, the 
DoD was able to set the boundaries of debate to exclude broader questions of what these 
developments might mean for the development of Al. It was assumed the technology should 
(or at least would) be used, it was simply a question of how it could. 

Identifying the same vast potential of Al technologies in defence as the DoD had, the 
Campaign was aghast. Instead of perceiving a technical issue regarding a technical solution, 
their framing of a fundamentally political challenge suggested an issue greater than an 
individual project or company. By drawing on the existing civil society movement around 
remote warfare and other disarmament campaigns, the Campaign was able to situate the 
development of Al here in a wider context of what could be considered permissible in 
warfare. From their perspective, the political problem thus deserved a political solution: a 
legal treaty retaining meaningful human control over the use of force. Precisely because it 
could occur, it should not be allowed. 

A few conclusions can be drawn from this process of forging alignments and 
rendering technical. First, there was a clear tension between the political and technical 
dimensions to the assemblage, which do not necessarily lie neatly with the political and 
technological actors. Second, the stakes were extremely high for all involved. Frictions 
between the elements were therefore obvious and profound. In general, these different 
elements were not necessarily interacting on precisely the same issues, at least not in their 
framing, which prompts us to consider how their differing interpretations can be reconciled 


(or not) between these assemblages. 
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5. Authorising Knowledge and Managing Failures 


“Authorising knowledge: assimilating science and containing critique.”"*' 


“Managing failures and contradictions: presenting failure as the outcome of rectifiable 


deficiencies; smoothing out contradictions...; devising compromises. ”!° 


Assemblages containing different elements with individual objectives can be characterised 
by their tensions as much as their commonalities. For this reason, the Military-Technological 
Complex worked hard to draw on expert knowledge to legitimise its actions and provide an 
appearance of cohesion. Understanding productive power here in the Foucualdian sense of 
“what can or cannot be said in a given configuration of knowledge,” attending to how these 
discourses are challenged helps reveal where authority lies."** For this reason, “fuzziness, 
adjustment and compromise are critical to holding the assemblages together.”"™ 

This section thus explores which discourses are authorised and excluded from the 
Military-Technological Complex, with reference to the experts and civil society critiques. 
From this, it will then examine how failures are managed, contradictions smoothed and 


compromises devised - or not, as the case may be. 


5.1. Authorising Knowledge 


5.1.1. Military- Technological Complex 
“Because we have different missions and different perspectives, sometimes we’re going to 
disagree,” admitted Defense Secretary Carter at Stanford when announcing the creation of 
the DIUx. He continued, “but | think that’s okay. Because being able to address tensions 
through our partnership is much better than not speaking at all.”"°° Aware of the challenges 
that lay ahead, Carter knew the DoD would have its work cut out reconciling tensions 


between the defense and tech community. Nevertheless, he was optimistic: “there can be 
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great ideas that come out of candid conversation.”'*° Project Maven, as a flagship effort, was 
certain to host some candid conversations. 

In a New York Times article titled “Eric Schmidt's Pentagon Offensive,” the former 
Alphabet Inc. Chairman was quoted as telling a four-star general, “you absolutely suck at 
machine learning... | could solve most of your problems.”'*’ Secretary Carter took Schmidt at 
his word and asked him to Chair the new Defense Innovation Board (DIB) in 2016.'°° Here 
he oversaw a body of “distinguished leaders with a track record of leading large, innovative 
organizations or conducting groundbreaking research” tasked with providing “independent 
advice to the Secretary of Defense and other senior leaders on catalyzing innovation in 
DoD.”"*’ Their expertise was supplemented by regular “listening sessions” with industry, 
where the DoD was “taking care to include not only experts... but also Al skeptics, DoD 
critics, and leading engineers who have never worked with the department before”’® Critique 
was thus welcomed, albeit within the confines of an officially-sanctioned forum. 

Later, in a testimony delivered to the House of Representatives Armed Services 
Committee, Schmidt endorsed Project Maven as “the most successful DoD effort to deliver 
Al to date.”"*' He noted, however, that the greatest successes the DoD had achieved in this 
domain - including Maven, the exemplar - were “largely developed outside of the mainstream 
DoD processes for developing and fielding capabilities.”"** Concluding his assessment, 
Schmidt declared “deeper focus, closer collaboration, more resources, and a sense of 
urgency are needed to solve problems of significance to the U.S.”"® Ethical concerns 
received no mention. Expertise thus confirmed that challenges here could be overcome by 
moving faster and collaborating closer. 

Just as tech experts went to the Pentagon, the Pentagon went to Silicon Valley. 


Speaking at an Nvidia GPU conference, Lt. Gen. Jack Shanahan declared, “let the machines 


186 Ibid. 

187 Conger, K., & Metz, C. (2020, May 3). ‘l Could Solve Most of Your Problems’: Eric Schmidt's 
Pentagon Offensive. New York Times. 

188 Ibid. 

189 Defense Innovation Board. (2020). Our Story. Retrieved from 
https://innovation.defense.gov/About1/ 

169 Defense Innovation Board. (2019). Public Listening Session. Retrieved from 
https://media.defense.gov/2019/Jun/07/2002142324/-1/-1/0/DIB_PUBLICLISTENINGSESSION_ 03.14 
.2019.PDF 6 

161 House Armed Services Committee. (2018). Statement of Dr. Eric Schmidt. Retrieved from 
https://docs.house.gov/meetings/AS/AS00/201804 17/108132/HHRG-115-AS00-Wstate-SchmidtE-201 
80417.pdf 

162 Ibid. 

163 Ibid. 


35 


do what machines do well, and let humans do what only humans can do.”'™ Similarly, 
researcher Gregory Allen credited the way Maven imitated tech sector techniques of iterative 
development for its success."® Incorporating and co-opting the knowledge and practices of 
the tech sector was arguably as important as the technology itself. To this end, Allen 
downplayed the technology in his framing of Project Maven as simply automating a 


bureaucratic task: 


“| have been among those advocating for the US military to increase its use of advanced Al 
technologies and to do so in a cautious and ethically conscious manner. Project Maven, 
which performs a non-safety-critical task that is not directly connected to the use of force, is 


exactly what | had hoped for.”'® 


Problematically, not everyone party to the assemblage considered Project Maven a 


“non-safety-critical task.” Not least some of those developing it. 


5.1.2. Civil Society Coalition 
Civil society was more than willing to fill in the areas where the DoD was reluctant to 
entertain debate. As discussed in the previous chapter, civil society had been investigating 
the legal, ethical and moral issues posed by LAWS since at least 2009. Project Maven 
reconfigured their discourse by bringing its technological elements to the fore. 

The Campaign members held a long tradition of campaigning and were 
well-rehearsed in its practices. Its coordinator Mary Wareham, for example, is co-laureate of 
the Nobel Peace Prize for her work on the International Campaign to Ban Landmines. When 
asked about the structure of the Campaign, member of the steering committee Frank Slijper 
drew parallels with similar coalitions formed to ban landmines (1997) and cluster munitions 
(2008)."*”’ Beyond the organisational structure, the experience of those involved helped 
frame this Campaign as the latest iteration in a strong heritage of arms control campaigns, 
allowing it to command significant influence from the outset. In Wareham’s words: “we’ve 


already established this principle that victim-activated weapons are not okay [via the treaty 
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banning antipersonnel landmines], it’s just being reiterated in different forms, and this is 
again what we’re coming back down to - uncontrollable effects.”'® 

Evidently, the Campaign shared certain characteristics with previous movements, but 
the mobilisation of tech workers in support was novel. In response to the news that their 
technology was being used for military purposes, more than 4,000 Google employees signed 
an open letter titled “our employer shouldn't be in the business of war.”"®’ By declaring that 
the contract “puts Google’s reputation at risk and stands in direct opposition to our core 
values,” they joined a well-established tradition of arms control campaigning, bringing with 
them valuable expertise about the technological elements, raising the profile of the issue and 
bolstering calls to ban LAWS. 

It is an obvious point but nevertheless worth stating: “conscientious tech objectors”’”° 
protesting against the military applications of their work were critically important to 
highlighting the technological issues with LAWS. In addition to the legal, political and ethical 
issues, there were many reasons to be concerned about the technology itself. Laura Nolan, 
a former Google employee who resigned in protest, succinctly explained: “we could build 
robots that can kill today. We cannot build a safe robot, that can’t be hacked, that works 
predictably in most or all situations, that is free of errors, and that can reliably manage the 
complexities involved in international law and the laws of war.”"”' This is particularly salient 
given the stark disparity in expertise between those building the technologies and those 
responsible for regulating them. In the opinion of Liz O’Sullivan, who worked at another 


Maven contractor: 


“There’s this significant divide; there’s the people who want the weapons... and the ones 
selling that technology, who claim that they can eliminate bias, and then the rest of civil 
society and boots-on-the-ground developers saying “wait a minute, | don’t think that sounds 
right.” But there’s a power imbalance so you hear the rhetoric of the people who have the 


power and not the ones who are trying to challenge that assumption... It's very dire the 
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imbalance of information and knowledge between the legislators and what we actually do on 


the industry side.” 


As well as holding expertise in the intricacies of these technologies, tech workers were 
conscious of the wider political issues. Despite Gregory Allen’s claim that Maven was “a 
non-safety-critical task that is not directly connected to the use of force,” Laura Nolan was 
under no illusions: “Maven was not killer robots but it was the automation of military 
surveillance [and] that is part of the kill chain.””? Political awareness of this sort was a highly 
significant critique given that Google is a multinational corporation. For Laura Nolan, who 
worked in Dublin, “the US military is not our military... nor is it a force we should 
automatically support as a matter of patriotism.”'* Frank Slijper concurred that for many 
multinational tech companies, leadership “will generally tend to be American - what is normal 
for them is not necessarily normal for the large majority of [international] employees.”'”* 
Proving Slijper’s point, Nolan’s opposition derived less from the technological reasons not to 
develop LAWS and more from an aversion to what she considered “a very unjust style of 
warfare and a very unjust series of wars” carried out by the USA.'”° 

In any case, the expertise offered by the tech workers who voiced their concerns 
publicly was greatly appreciated by the Campaign. Speaking at a UN side event, Nobel 
Peace Laureate and member of the Campaign Jody Williams declared that the controversy 
arising at Google over Project Maven “gave us something to then go back to Microsoft and 
Intel and the other major tech companies.”'”” An entire section of the Campaign website now 
calls upon tech workers to “hold your company, industry and peers accountable for the 
research they undertake and the customers they work with.”'” 

Mobilising these workers sets this campaign apart from its predecessors. Prior to the 
US election in 2016, according to Moira Weigal, “the tech industry was more likely to be the 
target of protests than it was to organise them.”'’? Today, however, Silicon Valley workers 


are shedding the “Californian Ideology” of free market economics and counter-culture 
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libertarianism."®° As one Google employee insisted: “libertarianism is the ethos of the leaders 
of these big tech companies, not the rank and file... we stood up because we believe 
workers should have a voice” (emphasis in original).'*' Many “conscientious tech objectors” 
who voiced concerns over Project Maven continue to play an active role in the Campaign 
today. Laura Nolan and Liz O’Sullivan left their jobs as a result of Project Maven and have 
since spoken at campaign events around the world, serving as “tech sector champions” to 
deepen engagement with industry. '* 

Further expert support for the Campaign came from UN Secretary General Antonio 
Guterres, who unambiguously told the Convention on Conventional Weapons (CCW) that 
LAWS are “politically unacceptable, morally repugnant and should be prohibited by 
international law.”"®? Because of this, Wareham likened Guterres to his predecessor Kofi 
Annan, a leading voice supporting the movement to ban landmines, indicating the UN 
Secretary-General is influential in raising the need for a treaty banning killer robots to be 
negotiated. '® 

Expertise and support from the tech sector helped to legitimise and strengthen the 
claims of the Campaign considerably. They were not, however, the goal in itself, which tech 
workers themselves were keenly aware of. In an address to an expert panel at the UN CCW, 


former Google engineer Amr Gabr declared: 


“Tech companies are massive private organizations that enforce decisions and outcomes 
through code. Not laws, not deliberation, not police, not militaries... However when these 
decisions fail to protect human lives it is governments and regulatory bodies who will also be 
held accountable and responsible. This is where the public interest, employee interests, state, 


and UN interests align in my opinion.”'® 


Tech workers and other Campaign allies, such as Antonio Guterres, thus helped play a 
legitimising function. By reflecting a broad and expert support base, emphasising 
technological concerns and embracing the political arguments, they lent credibility to the 


cause. 
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5.2. Managing Failures and Contradictions 


Discourses advanced and knowledge authorised by the Military-Technological Complex 
were at direct odds with the Campaign to Stop Killer Robots and their newfound allies in the 
tech community. How these would be reconciled - or not - in the case of Project Maven is 


instructive in revealing the exercise of power by each configuration. 


5.2.1. Military- Technological Complex 
A very public struggle over Maven followed the open letter signed by Google employees. In 
fact, this fallout was perhaps the defining characteristic that makes this case so valuable for 
revealing the resilience of the Military-Technological Complex. Through the response to 
these protests we see how the assemblage seeks to "present failures as the outcome of 
rectifiable deficiencies in technique, to smooth out contradictions and to devise 
compromises.”'®° 

When its workers first complained, the CEO of Google’s cloud businesses (those 
responsible for Maven) Diane Greene tried to quell dissent by emphasising the 
“non-offensive” nature of this work.'®’ Perhaps unsurprisingly, this did little to alleviate 
concerns. As the controversy grew, management recognised ethical concerns but 
highlighted their complexity, asserted that ethical principles would guide future work, and 
re-emphasised the limited scope of this project.'’*® Recourse to technique rather than 
substance in this manner had limited impact. 

Eventually, sustained pressure from Google workers (and wider civil society) 
prompted Greene to declare Google would not seek renewal for its Project Maven contract. 
Declaring that “Google Cloud’s initiatives around Al are relatively new” and that “Al is a 
disruptive technology that allows for unprecedented uses,” Greene emphasised its novelty in 
her decision, essentially framing it as a learning process.'®? Rather than constituting an 
outright “failure” in the Military-Technological Complex, however, Google’s withdrawal was 
carefully managed. Greene insisted that Google would not cancel its contract entirely, as 
some had demanded, but simply not renew it: “I would like to be unequivocal that Google 


Cloud honors its contracts.”'°° Even here, in perhaps the most noteworthy failure of the 
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Military-Technological Complex as an assemblage, Google devised a mutually-convenient 
compromise between its own employees and the DoD. 

The DoD, for its part, steadfastly emphasised the successes of this project and its 
wider significance beyond immediate deliverables. During the early stages of Maven (prior to 
controversy breaking out at Google), Lt. Gen. Shanahan downplayed the hurdles 
encountered as typical of “any disruptive effort within the defense community.”"*' Even much 
later, after Google had cut ties with the Pentagon, Shanahan re-asserted that Maven was a 
“pathfinder” for producing a “product delivery pipeline.”'** Looking to the future in this manner 
served to frame the split as a learning experience for the next project, smoothing over this 


failure as transient. 


5.2.2. Civil Society Coalition 

In the case of Project Maven, the Campaign to Stop Killer Robots had no failure to manage. 
Successfully pressuring Google into not renewing its Project Maven contract, through 
sustained and coordinated action with tech workers, constituted a resounding victory. 
Problematically, however, the reassembly of the Military-Technological Complex (detailed 
further in the next chapter) evidenced its resilience. Google’s departure from the 
Military-Technological Complex was only one battle in a larger war. Victory was not to be 
taken for granted. 

The Campaign recognised there was more work still to be done. Responding to the 
news Google would not seek renewal for its Project Maven contract, it welcomed the 
development but invited other “responsible companies” to “publicly support the increasing 
calls for states to urgently negotiate a new treaty to prohibit fully autonomous weapons.”'®? 
Instead of managing failures, civil society was managing success. Perhaps more accurately, 
it was seeking to manage expectations following success. Sceptical of the principles Google 
had announced, Mary Wareham commented at a UN side event: “Political declarations, 
principles, promises of transparency, codes of conduct, we've heard this all before. For us, 


we need binding international law in the form of a new treaty.” Rather than devising 
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compromises that smooth over the contradictions, the Campaign drew attention to the work 


still to be done. 


5.3. Reasserting the will to govern 


The containment of critique and management of failures in an assemblage ultimately 
reasserts the will to govern.'® In this regard, Project Maven served as something of a 
lightning rod for critique. 

As the visible face of an often-intangible phenomenon, Project Maven became a 
staging ground in a much larger battle, revealing a deep tension between the compulsory, 
material powers of the DoD and the productive, social powers of civil society. Its supporters 
praised it as a model to emulate for future public-private cooperation, with immense material 
rewards, while its detractors attacked it as an archetypal example of the ethical risks posed 
by these technologies. The contestation and authorisation of knowledge here can therefore 
be considered profoundly important to future applications of this technology in this domain. 

For the DoD, the near-instant successes of Maven were warmly received. A welcome 
shakeup to the traditionally convoluted and glacial processes of defence contracting, it 
reaffirmed to many not only the critical importance of close collaboration with the tech sector 
but also the potential rewards. The role of the DIB - and particularly its chairperson Eric 
Schmidt - in championing the importance of cooperation in this regard has been vital to 
reasserting the DoD’s ability to direct the development of this technology. Expert knowledge 
was thus strategically used to downplay the political and foreground the technical in the 
Military- Technological Complex. 

Reverse dynamics are visible among civil society actors. Those involved in the 
Campaign from its early days benefited from a rich tradition and body of knowledge in 
disarmament campaigns. Tech workers presented a novel source of support after Maven 
triggered their mobilisation and prompted wider engagement with the ethics of their work. A 
combination of the Campaign’s pre-existing expertise in international diplomacy and tech 
workers’ technological expertise has established a force to be reckoned with. As a result, the 
Campaign has strategically drawn upon expert knowledge to frame this as a political and 
ethical issue first and foremost. 

Differences in what was accepted as expert knowledge among different members of 
the assemblage ultimately proved untenable. A fracture was inevitable. The storm emerging 


out of Project Maven laid bare the fragility of the Military-Technological Complex. Alignment 
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between these actors was revealed to be weak and the early successes they had achieved 


unstable. We turn now to what followed. 
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6. Antipolitics and Reassembly 


“Antipolitics: reposing political questions as matters of technique; closing down debate about 
how and what to govern...; encouraging citizens to engage in debate while limiting the 


agenda." 


“Reassembling: grafting on new elements and reworking old ones; deploying existing 


discourses to new ends; transposing the meanings of key terms.”°” 


Practices of antipolitics and reassembly show how assemblages evolve into something new. 
From the failures and contradictions identified in the previous chapter, it was clear that the 
previous configuration was unsustainable and change was inevitable. Fortunately, as 
mutability and fluidity are fundamental characteristics of any assemblage, this heuristic tool 
is well-equipped to account for such change in Project Maven. Reassembly is precisely that: 
a reconfiguration. 

This section therefore answers how political questions are re-posed as matters of 
technique and the resulting, re-assembled form. Given the overarching goal of ‘locating 
power in complexity’, we can expect this rearrangement to reflect the material and social 


effects of compulsory and productive power within the assemblages. 


6.1. Arntipolitics 


6.1.1. Military Technological Complex 
When Google cut ties with the DoD, the Military-Technological Complex did not cease to 
exist. Al technologies remained a top priority for the DoD and its work in developing and 
procuring them still required legitimising and justifying, albeit with new discourses. Such a 
long-term perspective was evident in Lt. Gen. Shanahan’s description of Google’s departure 
as a “canary in a coal mine,” expressing relief “that it happened when it did as opposed to on 
the verge of a conflict or a crisis where we're asking for help.”'%® In this regard, Project 


Maven presented a steep learning curve for the DoD. Controversy surrounding it raised the 
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profile of the issue among tech workers, civil society, and the broader public, but it also 
taught the DoD how to avoid a repeat. 

Several Military-Technological Complex proponents attempted to depoliticise this 
issue by invoking notions of patriotic duty. Michael Bloomberg deemed Google’s decision “a 
defeat for US national security [and] patriotism.”'°? Similarly, Amazon CEO Jeff Bezos asked, 
“do you want a strong national defense or don’t you? | think you do. So we have to support 
that... if big tech is going to turn their backs on the Department of Defense, this country’s in 
trouble.”2°° Meanwhile at Microsoft, facing similar controversies over its provision of Hololens 
Augmented Reality headsets to the US Army, CEO Brad Smith declared, “we believe in the 
strong defense of the United States and we want the people who defend it to have access to 
the nation’s best technology.”2"' Eric Schmidt even described Lt. Gen. Shanahan as “a real 
American hero” for his work on Maven.” Chairman of the Joint Chiefs of Staff Joseph 
Dunford concurred in a succinct message to Silicon Valley: “hey, we’re the good guys.”*° 

Invoking a patriotic discourse can be considered an attempt at closing down debate 
surrounding the development of Al for military purposes. Military service in the US has long 
been valorised as “the definitive demonstration of citizenship, the most heroic, the most 
dangerous, and the most selfless.”* As previously noted, such arguments may find little 
resonance among international employees for whom the US military is not their military. As 
one Google worker insisted, even those who did not support cancelling the Maven contract 
saw this as a weak argument, as Google is an international company, and of those who did, 
many come from “regions of the world where the American military has been extremely 
destructive.”*°° There was no escaping the strength of feeling among those workers now 
aligned with civil society. Its original elements could no longer cohere and a reassembly was 


inevitable. 
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6.1.2. Civil Society Coalition 

In June 2018, Google aligned itself with civil society with the announcement that it would not 
renew its Project Maven contract. CEO Diane Greene acknowledged that Google would not 
pursue the contract again because of the backlash the company had experienced.” For 
those conscientious tech objectors who blew the whistle on Project Maven, Greene’s 
decision appeared a resounding victory. A week later, Google announced ethical principles 
to guide its future work in this domain.” These included a commitment not to design Al for 
“weapons or other technologies whose principal purpose or implementation is to cause or 
directly facilitate injury to people” and “technologies whose purpose contravenes widely 
accepted principles of international law and human rights.”””° 

Ethical principles reposed political questions as a matter of technique by providing 
clear guidelines for future work. Despite Greene’s insistence that Google’s contribution to 
Maven was for “non-offensive purposes,” it would be difficult to reconcile a commitment not 
to work on weapons with any future involvement in the Algorithmic Warfare Cross-Functional 
Team (as Maven is formally known). Similarly, deferring to “widely accepted international 
human law and human rights” intended to circumvent ethical and political debates around 
future work in favour of a neutral arbiter. Notably, the list of applications Google would not 
work on included a clause that “as our experience in this space deepens, this list may 
evolve,” leaving the door open for further debate in the future.°° 

Not all of civil society was satisfied that ethical principles would be sufficient. These 
principles were, after all, written and adopted by Google unilaterally. No regulatory 
mechanisms or external oversight were included, leaving it to Google to interpret and 
implement them as they saw appropriate. In the absence of any external control, principles 
like “be socially beneficial” and “avoid creating or reinforcing unfair bias” appear ambiguous 


and, to some, of dubious value. Laura Nolan was extremely sceptical: 


“A lot of these companies come out with ethics policies as a cheap way to make it look like 
they're doing something. Google’s one is so vague that it’s very hard to tell what would and 


would not be allowed under it... And no-one can really see how they are implementing that 
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policy. Policy that doesn’t really have any process or oversight or transparency around it - 


what's the point?”2"° 


As Frank Slijper concurred, “having a policy is one thing and implementing it is another.”2"" 
While a commitment not to develop weapons sounds unequivocal on first reading, Slijper 
noted that, “the line between what is lethal or potentially lethal or non-lethal is not a black 
and white area, there is always a grey area in between.”*”? Similarly, reference to 
international law and human rights in these principles appears laudable, but as Peter Asaro 
observed, “the international norms surrounding... drone surveillance are all contested and 
debated in the international sphere.”?"% 

The scepticism invited by the ambiguity of these principles was vindicated when Kent 
Walker, Google’s Senior Vice President for Global Affairs, later declared that the decision 
not to renew their Maven contract was “focused on a discrete contract” and should not be 
interpreted as “a broader statement about our willingness or our history of working with the 
Department of Defense.”*"* Recourse to technique in the form of vague principles may 
therefore have limited impact in closing down debate, as the door remained open to future 
DoD work. As Laura Nolan summarised: “self-regulation is no regulation.”?"° 

For the Campaign, Google’s departure from Project Maven was just one victory ina 
much larger battle. Ultimately, for Mary Wareham, “the companies are not within the 
immediate scope of the Campaign... we’re going after governments as governments are the 
ones that negotiate, adopt and regulate through [a] multilateral instrument.”*"° Any ethical 
principles are thus insufficient to quell their concerns. From their perspective, in the absence 


of a legally-binding treaty banning fully autonomous weapons, the issue remains unresolved. 


6.2. Reassembly 


6.2.1. Military-Technological Complex 
The departure of Google from Project Maven constituted a reassembly of the 


Military-Technological Complex. Despite the negative publicity associated with Maven, 
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however, many companies were willing to pick up where Google left off. New companies and 
discourses were incorporated into the assemblage and its form was altered. 

Clarifai was another tech company contracted to work on Maven. Its involvement 
became public shortly after Google decided not to renew its contract. Notably, Clarifais’s 
CEO Matthew Zeiler was unfazed by the bad press which followed, declaring “the goal for 
our contribution to Project Maven - to save the lives of soldiers and civilians alike - is 
unequivocally aligned with our mission [as a company]”*"” Embracing the patriotic discourse, 
Clarifai became a visible element of the Military-Technological Complex. In Zeiler’s words, 
“we believe in putting our resources toward society's best interests, and that includes 
America’s security.”*" 

Around the same time, data analytics company Palantir reportedly took on the work 
left by Google in Maven. Like Clarifai, it had no qualms delivering such controversial work. In 
fact, co-founder Peter Thiel called upon the FBI and CIA to investigate Google for its 
“treasonous” decision not to work on Maven.?'* Three Palantir executives went on to form 
Anduril, a tech company focused on “remaking” defence contracting “by incorporating the 
latest innovations of Silicon Valley into warfighting technology.”“”° In addition to the former 
Palantir executives, Anduril boasts of an “elite team of industry experts” hailing from 
high-profile tech companies, such as SpaceX, Tesla and Google, as well as traditional 
defence contractors, such as General Atomics. 

Anduril co-founders Palmer Luckey and Trae Stephens vocally criticised Google's 
decision not to renew its Maven contract, insisting “if tech companies want to promote 
peace, they should stand with, not against, the United States’ defense community.””*' 
Notably, however, they recognised and even shared many of the concerns of those 


conscientious tech objectors at Google, but disagreed on the appropriate solution: 


“We agree with many others that the decision to take a human life should not be made 
without human direction. But an essential part of ensuring that technologies are used ethically 


is ensuring that the terms are not dictated by authoritarian regimes. For the United States to 
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set ethical norms and assert a moral high ground, it must first hold the technological high 


ground.”222 


The following year, in March 2019, Anduril won a Maven contract.”3 Their involvement in 
this drone surveillance technology resonates with Luckey’s vision of the future of warfare, 
one in which soldiers are “superheroes” with “perfect omniscience over their area of 
operations, where they know where every enemy is, every friend is, every asset is.”4 For 
better or worse, Anduril has a reputation in this domain, having previously developed an 
autonomous border surveillance system for US Customs and Border Protection.” Evidently, 
it is comfortable adopting and perpetuating a ‘patriotic’ discourse to justify its defence work. 
Changes on the industry side of the Military-Technological Complex were mirrored in 
the DoD itself. True to his vision of Maven as the “spark that kindles the flame front of 
artificial intelligence across the rest of the [Defense] Department,”*° Lt. Gen. Shanahan went 
on to lead the Pentagon's Joint Artificial Intelligence Centre (JAIC) when it was created in 
June 2018. In its founding memo, Shanahan wrote that the JAIC was established for 
“accelerating the delivery of Al-enabled capabilities, scaling the Department-wide impact of 
Al, and synchronizing DoD Al activities to expand joint force advantages.” Essentially, the 
JAIC intended to do for the entire DoD what Project Maven had done for drone footage 
analysis.**’ In doing so, the JAIC was fulfilling a vision proposed by Lt. Gen. Shanahan a 
year prior: that the DoD “should never buy another technological platform without artificial 
intelligence capabilities baked into it.”?”8 
Now that the spark of Project Maven has kindled the flame of the JAIC, the DoD is 


rolling out its Al technologies for applications beyond the battlefield. These include tracking 
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wildfires, hurricanes and assisting with humanitarian disaster relief.??? Recent news reports 
suggest the tools developed as part of Maven are being repurposed towards an algorithmic 
surveillance system for tracking the spread of covid-19.7%° Clearly, the Al technologies 
involved can be repurposed towards different ends than those originally envisaged - a core 
concern of those Google workers protesting. Implementing Maven technologies in more 
benign circumstances such as these suggests the DoD is seeking to repair the negative 
reputation it has earned for its Al integration. If effective, this may improve the likelihood of 
tech companies being willing to work with it in the future. According to Shanahan’s own 


description of JAIC work in humanitarian and disaster response: 


“One of the most important benefits of this NMI [National Mission Initiative] is that it is an 
inspiring, societally-beneficial, life-saving mission that is not only whole-of-government but 
whole-of-society... It offers a unique opportunity to combine DoD efforts with industry and 
academia in a new type of public-private endeavor to operationalize Al to solve our most 


challenging problems.”?*" 


Old technologies are thus reworked to new ends but via the same means - enlisting tech 
companies. As a dynamic and ongoing process, it requires constant work to legitimise and 


justify. For this reason, civil society remains highly relevant. 


6.2.2. Civil Society Coalition 

Beyond the decision taken by Google not to renew its Maven contract, the impact of civil 
society on the reconfigured assemblage was striking. As previously noted, even those 
contractors brought in to replace Google in Project Maven conceded that life and death 
decisions in war should not be delegated to machines. Even though Google formally ‘left’ the 
Military-Technological Complex and the Campaign has moved on to bigger objectives, their 
influence endured. 

Following the controversy that arose over Project Maven, a discourse of ethics is now 
playing a much greater role in the Military-Technological Complex. “To underscore our focus 


on ethics, humanitarian considerations, and... Al safety,” announced Lt. Gen. Shanahan in 
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2019, “JAIC is working closely with the Defense Innovation Board (DIB) to foster a broad 
dialogue and provide input into the development of Al principles for defense.”**? In 
September 2019, Shanahan announced that the JAIC would be hiring a dedicated ethicist 


for this purpose.?*° 


It remains to be seen how meaningful these actions will be longer-term, 
but the incorporation of such rhetoric into the Military-Technological Complex demonstrates 
the lasting influence of civil society in its newly-reassembled form. 

While the Campaign continues to encourage tech workers to support its cause and 
question the uses of their work, it faces a new challenge in the rise of “bespoke” contractors 
like Anduril. A large part of the success achieved in mobilising workers at Google comes 
down to the fact that those workers were not necessarily aware of their company’s defence 
contracts: “most people assumed, or had been told, that Google doesn’t do military work,” 
according to Laura Nolan.*** As Frank Slijper elaborates, “if you apply for a job with 
Raytheon, you know you’re going to build bombs and missiles, but if you apply for a job with 
Google or Amazon then probably you have a bit of a different idea of what your work will 
look like.”2°° External pressure on these companies is more effective for the same reason. At 
Google, according to Slijper, “the majority of their income comes through people like you and 
me, who ensure massive interest from advertisers... and so they are - especially since 
Project Maven - very well aware of any potential [public] backlash.”*°° Conversely, however, 
for defence-tech companies like Anduril with no commercial businesses beyond Government 
contracting, public pressure may hold less sway. 

Pressuring tech companies alone remains as impractical as it is undesirable for the 
Campaign, for whom the overarching goal remains a legally binding treaty. Tech sector 
support is welcome but their sights are set elsewhere: the international community. As 
Campaign coordinator Mary Wareham noted, “we're not attempting to create a treaty by only 
working through the tech sector.”*°’ Governments are the ones who will ultimately decide on 
the fate of LAWS, according to her, “because that’s who gets to launch, negotiate, adopt, 
sign, ratify and implement international treaties.”?%° 
In the reassembled form of the Military-Technological Complex, civil society (and in 


particular The Campaign) perhaps has less direct influence. The ease with which Google 
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was replaced proves the limits of their power in this regard. Their effect - and, similarly, that 
of Google - is thus more of a ‘legacy’ insofar as it has prompted a turn in the DoD towards 
prioritising the ethics of these technologies and responsible development, particularly 


vis-a-vis great power rivals. Of course, however, the durability of this remains to be seen. 


6.3. Rewiring the Military- Technological Complex 


Examining the practices of antipolitics and reassembly in this context reveal a new form for 
the Military-Technological Complex. Bridging the divide between the material power of the 
DoD and social power of civil society, a new breed of “defence-tech” companies like Anduril 
have carved out a niche as bespoke contractors. Having drawn engineering talent from both 
traditional defence companies and Silicon Valley, they exclusively but openly court defence 
contracts, invoking a discourse of patriotism without risking the alienation of a wider 
consumer market. The ease and speed with which they began providing this work speaks to 
the significant power of the DoD and, in turn, the resilience of the Military-Technological 
Complex as an assemblage. 

Underlying this entire process is a constant tension between the predominantly 
compulsory power of the DoD and the predominantly productive power of civil society. 
Sizeable contracts provide a compelling material incentive for many tech companies to 
provide the technologies sought by the DoD, but are in conflict with the ethical pitfalls 
publicised by civil society. In some ways, each mimics the other. As the DoD has adopted a 
discourse of patriotism to assuage ethical concerns, civil society has underlined the material 
risk to businesses with large consumer markets seen to be enabling these developments in 
warfare. 

As a result, the civil-military industrial divide appears more porous than ever, raising 
valuable questions about contemporary warfare. Project Maven exemplifies a certain fluidity, 
in that its technologies were originally developed for the consumer market, weaponised by 
the DoD for drone footage analysis, and are now being “re-legitimised” through their transfer 
back into the civilian sphere for humanitarian and public health purposes. Such a process 
arguably reflects a late-modern - or liquid modern - blurring of the boundaries between war 
and peace, military and civilian, public and private. 

A blurring civil-military distinction in this regard must be understood in the context of 


a geographically and temporally expanding militarism observed in the rise of “unending’**° 


238 Duffield, M. (2007). Development, Security and Unending War: Governing the World of Peoples. 
Polity. 
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and “everywhere” wars. Similarly, in Graham’s conceptualisation of the “battlespace” of 


cities, he observes it “has no front and no back and no start or end... the concept... thus 
works by collapsing conventional military-civilian binaries.”**' Common to these suggestions 
that conventional ties between war, space and time are coming undone, however, is a focus 
on their practice.“* The ascendancy of the Military-Technological Complex suggests that the 
nebulous expansion of warfare across space and time not only pertains to how it is practiced 
but also how its means are developed and procured. It is not a stretch to suggest that this 
expansion, in turn, risks normalising new imaginations of violence.“? And given that 
“violence needs to be imagined in order to be carried out,” one wonders where this 
normalisation may lead.2“* 

Conceptualising the Military-Technological Complex as an assemblage is particularly 
appropriate in the context of liquid modernity. Absent of rigid, ‘arboretic’ structures, it takes 
shape in a manner consistent with Bauman’s interpretation of sociality, “spreading 
rhizomically and sprouting formations of varying degree of durability, but invariably unstable, 


hotly contested and devoid of foundation to rely on.” Fluidity of formation and re-formation 


in this regard, however, should be regarded as “a feature of state power, not a bug.””“° 


Instead of the state standing separately to the private sphere, it is intimately embedded 


within it and continually reproducing its power through it.”*’ 


Lastly, directly relevant to this “undoing” of conventional warfare and fluidity of the 
military-civilian divide is Foucualt’s ‘boomerang’, which suggests the practices - and 
technologies - of peripheral subjugation inevitably return to the centre.*? Or, as Marx 


suggested, the scenario of “war [being] developed earlier than peace.””*? Indeed, many have 


40 Gregory, D. (2011). The everywhere war. Geographical Journal, 177(3), 238-250. 
https://doi.org/10.1111/j.1475-4959.2011.00426.x 
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noted the ways in which the United States’ war on terror has already “come home to roost” in 
the form of widespread surveillance”? and militarised police forces.”°' One of the most overt 


and egregious examples can be seen in the recent use of an MQ-9 Predator drone for 


252 


surveilling Black Lives Matter protests in Minneapolis.** It is not known whether this drone 


was utilising the technologies developed in Project Maven. But nor is it hard to imagine. 


50 Moltke, H. (2019, May 29). Mission Creep: How the NSA’s Game-Changing Targeting System Built 
for lraq and Afghanistan Ended Up On The Mexico Border. The Intercept. Retrieved from 
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251 Kommena, N., & Kirk, A. (2020, June 5). Why are some US police forces equipped like military 
units? The Guardian. Retrieved from 
https://Awww.thequardian.com/world/2020/jun/05/why-are-some-us-police-forces-equipped-like-military 
-units 

252 Koebler, J., Cox, J., & Pearson, J. (2020, May 29). Customs and Border Protection is Flying a 
Predator Drone Over Minneapolis. Vice. Retrieved from 
https://www.vice.com/en_us/article/5dzbe3/customs-and-border-protection-predator-drone-minneapoli 
s-george-floyd 


54 


7. Conclusion 


“There is nothing predestined about the impact of Al,” according to Google’s guidelines for 
the Responsible Development of Al.”°? Indeed much of the story of Al in defence remains to 
be written, however it is hoped that this research has not only addressed an important 
research gap but also emphasised the importance of grappling with these issues. 

Just as there is nothing predestined about the impact of Al, there was nothing 
predestined about the way Project Maven played out. The formation of a 
Military-Technological Complex here has been bitterly contested, the product of a lengthy 
process which began well before controversies arose in 2017. Since at least 2014, the DoD 
has sought to develop, purchase and field Al technologies to maintain superiority. 
Recognising it could not do so alone, it went to considerable effort to establish partnerships 
and facilitate cooperation with consumer technology companies at the cutting edge of Al 
development. In addition, it carefully cultivated amenable, expert opinions to legitimise these 
moves, framed its failures as learning experiences, and adapted its methods and advanced 
new discourses - particularly concerning patriotism - to justify its measures. A continual 
strategy of downplaying the political and emphasising the technical has been deployed 
throughout, lending the Military-Technological Complex an impression of coherence. All the 
while, this has been enabled by the immense, compulsory power granted by the material 
resources of the DoD, and structural relations they engender. 

For proof that there was nothing predetermined about Project Maven, consider the 
tireless efforts of civil society in raising the alarm over these developments. A broad coalition 
of different organisations, unified by a shared concern about the loss of human control from 
warfare, have highlighted to tech workers the ethical repercussions of their defence work. In 
turn, many tech workers have bolstered the cause of the Campaign by providing valuable 
expert input. Although the tech sector is neither the goal of the campaign nor the means to 
achieve it, the decision of Google not to renew its Project Maven contract has had a ‘legacy’ 
effect in the Military-Technological Complex, forcing others to recognise and address 
questions of ethics in this work. In this regard, civil society had an important role in 
exercising its productive power through systems of knowledge and meaning pertaining to 
ethical concerns about LAWS. 

In addressing these issues, this thesis has shed light on the development of LAWS 


and contributed to contemporary discussions surrounding remote warfare. There remains, 


253 Google. (2018). Responsible Development of Al. Retrieved from 
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however, work to be done. In particular, further study into the role of new, dedicated 
defence-tech contractors who appear to have fewer qualms about the ethics of this work is 
urgently needed. Additionally, comparative studies into similar dynamics in other countries 
involved in the development of LAWS (e.g. Russia, China) may bear interesting results. 
More generally, however, as Al technologies are set to proliferate and affect much more than 
just defence, there is an urgent need to interrogate their development and question how they 


may cause harm. Understanding this, after all, is the first step to countering it. 
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